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Abstract 


Building on prior research by two of the present authors, which uses lottery- like features in New 
York City’s high school admissions process to rigorously demonstrate that new small public 
high schools in the district are markedly improving graduation prospects for disadvantaged 
students, the present paper demonstrates that these graduation benefits do not come at the cost 
of higher expenditures per graduate. The basis for these findings are two cost comparisons: (1) a 
“descriptive” comparison of per-pupil operating costs for the new small high schools with those 
for all other district high schools, and (2) an “experimental” comparison of per-pupil operating 
costs for the new small high schools with those attended by their control group counterparts. 
The descriptive comparison demonstrates that the new small schools spend a little more per 
pupil than the average district high school and a lot more than the largest of these other schools. 
By contrast, results of the experimental comparisons together with previous findings of two of 
the present authors about the substantial positive effects of the new small schools on high school 
graduation rates indicate that the cost per high school graduate is substantially lower for the 
small-school enrollees than for their control group counterparts. This seemingly counterintui- 
tive result occurs because control group counterparts (1) attend high schools with annual per- 
pupil costs that are about the same as those for the new small schools, (2) are more likely to 
attend a fifth year of high school because they do not graduate in four years, and (3) are less 
likely to graduate from high school at all. 
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1. Introduction 

As opportunities to exploit random assignment to treatments and access to longitudinal, student- 
level data have expanded in recent years, evaluations of educational interventions have grown in 
sophistication. Most of this evaluation work has focused on estimating intervention effects on 
student outcomes — that is, the benefits of interventions. Careful attention to the costs of 
interventions has occurred much less frequently, and rigorous methods for estimating such costs 
have rarely been used. The present study takes advantage of a unique opportunity to estimate 
the costs and benefits of a large-scale educational intervention using the same rigorous methods 
for both analyses and thereby inform key policy questions. 

Specifically, this study examines the costs of newly created, academically nonselective 
small public high schools of choice (SSCs) in New York City. A previous evaluation of these 
schools, which exploited randomly assigned high school admissions, found that students who 
attend an SSC are 9.5 percentage points more likely to graduate from high school than they 
would have been had they gone to the other New York City public high schools attended by 
their control group counterparts (Bloom and Untennan, 2014). 1 The existing literature, howev- 
er, suggests that small high schools might cost more to operate than larger high schools because 
small schools cannot exploit economies of scale (Andrews, Duncombe, and Yinger, 2002; 
Stiefel et al., 2009). The present paper helps to complete the evaluation of SSCs by considering 
the school costs that were incurred to produce the previously estimated graduation benefits, and 
its analyses help to inform the debate over small schools that has been ongoing for more than a 
decade (Wasley et al., 2000). The paper also attempts to promote better methods for assessing 
the costs of educational interventions by presenting a new approach to estimating these costs 
and identifying the policy questions that such analyses can inform. 

The sections that follow present two complementary sets of analyses: The first set 
(which we call “descriptive” comparisons) compares per-pupil expenditures and other 
measures of resource use by SSCs with per-pupil expenditures and resource use by other New 
York City high schools (defined later), on average overall and by school size. These compari- 
sons (1) confirm differences between the costs of small and large high schools that have been 
reported by other researchers; (2) explore some reasons why SSC costs differ from those 
of other high schools operated by the New York City Department of Education (NYCDOE); 
(3) provide insights into the potential cost implications of replacing existing large public high 
schools in New York City with additional SSCs; and (4) help to contextualize the present 


'The 9.5 percentage point effect of SSCs on graduation rates reported by Bloom and Unterman (2014) was 
based on data for three annual cohorts of entering ninth graders. A forthcoming analysis that adds data for 
another student cohort reports a 9.4 percentage point effect (Unterman, 2014). 
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findings by enabling other researchers to compare high school costs in New Y ork City with 
those elsewhere. 

The second set of analyses (which we call “experimental” comparisons) employs the 
experimental/instrumental variables framework used by Bloom and Untennan (2014) to 
estimate the effect on graduation rates of attending SSCs (relative to the New York City public 
high schools attended by the control group counterparts of SSC enrollees) in order to estimate 
the difference in resources devoted to those treatment and control group students. These 
analyses explicitly compare the costs of SSCs with the corresponding costs of their “counterfac- 
tual counterparts.” Doing so represents an innovative approach to rigorously addressing the 
question: How do the differential costs of SSCs compare with their differential effectiveness? 

Results of our descriptive comparisons indicate that (1) annual operating expenditures 
per pupil for SSCs are modestly higher than those for the typical NYCDOE high school and 
substantially higher than those for especially large NYCDOE high schools (with more than 
2,000 students); and (2) this conclusion does not change when operating expenditures are 
adjusted statistically for school differences in teacher salaries and student composition. 

By contrast, results of our experimental comparisons together with previous findings 
about the substantial positive effects of SSCs on high school graduation rates (Bloom and 
Untennan, 2014) indicate that the cost per high school graduate is substantially lower for SSC 
enrollees than for their control group counterparts. This seemingly counterintuitive result 
occurs because their control group counterparts (1) attend high schools with annual per-pupil 
costs that are about the same as those for SSCs, (2) are more likely to attend a fifth year of high 
school, and (3) are less likely to graduate from high school. 

These findings provide a compelling case that the benefits of existing SSCs — higher 
graduation rates and fewer students in need of a fifth year of high school — are substantially 
greater than their costs. Thus from a cost-benefit perspective, New York City’s SSCs should 
be continued. The findings also provide suggestive evidence that expanding SSCs in New 
York City or implementing them in other large urban districts might be a cost-effective way 
to improve educational opportunities for additional low-income disadvantaged students. 
However, as discussed later, decisions about expanding small schools in New York City or 
elsewhere involve additional considerations that are beyond the scope of the present analysis, 
such as which existing high schools SSCs would replace and which students these new 
schools would serve. 

The remainder of this paper is organized as follows: Section 2 provides background on 
the New York City small high school reforms and the small high schools that we study. Sec- 
tions 3 and 4 describe the sample, data, and cost measures that we use. Sections 5 and 6 present 
the results of the two complementary cost analyses that we conducted. Section 7 considers the 
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implications of several additional sources of costs that could not be fully accounted for, and 
Section 8 describes the implications of our analyses for policy. 


2. Background on New York City’s Small Schools of Choice 
and Their Likely Costs 

As part of sweeping high school reforms, more than 200 new, small public high schools were 
created in New York City between 2002 and 2008. Although some of these schools screen 
students based on their academic backgrounds and others focus specifically on students who are 
overage and/or undercredited, the majority are academically nonselective and provide options 
for students regardless of their prior academic attainment or achievement. We refer to these 
schools as New York City’s “Small Schools of Choice” (SSCs). Most of these new small 
schools serve concentrations of disadvantaged students of color and are located in buildings 
where larger high schools with low levels of student achievement had been closed. 

In a recent evaluation of New York City’s SSCs, Bloom and Untennan (2014) exploit- 
ed randomized school admission lotteries created by New York City’s high school application 
processing system to estimate the effects of enrolling in an S SC on a variety of student out- 
comes, including high school graduation rates. Based on a sample of students who participated 
in ninth-grade admission lotteries for 84 SSCs in the fall of 2005, 2006, or 2007, these analyses 
indicate that attending an SSC instead of another New York City public high school increased 
four-year graduation rates by 9.5 percentage points (from 60.9 percent to 70.4 percent) and five- 
year graduation rates by 8.9 percentage points (from 64.7 percent to 73.6 percent). Similar gains 
were documented for a wide range of subgroups, including students with eighth-grade state test 
scores that were below or above grade-level proficiency, students from low-income families, 
and students from different demographic groups (Bloom, Levy Thompson, and Untennan, 
2010; Bloom and Untennan, 2012; Bloom and Untennan, 2014). 

However, a complete assessment of SSCs requires infomiation not only on their bene- 
fits, but also on their costs, which the existing literature suggests might be higher than average. 
SSCs are unifonnly small in size, serving roughly 100-120 students per grade, and are consider- 
ably smaller on average than other New York City public high schools. One might expect costs 
to decline as school size increases because fixed costs associated with indivisible inputs such as 
administrative and student support personnel, teachers in specialized subjects, and gymnasiums, 
science labs, libraries, auditoriums, and cafeterias can be spread over more students in large 
schools. Especially large schools, however, might create managerial challenges such that the 
effectiveness of fixed resources might decline after the number of students exceeds a certain 
point, thereby increasing costs (Andrews, Duncombe, and Yinger, 2002; Stiefel et al., 2009). 
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Empirical studies of the relationship between school size and costs have typically 
tried to estimate the marginal effect of increased enrollment on expenditures, controlling for 
student outcomes, input prices (particularly teacher wages), and differences between schools 
in students’ educational needs. Challenges in estimating these models include measuring 
outcomes, input prices, and student needs; accounting for the simultaneous determination of 
student outcomes and school expenditures; and controlling for efficiency. 2 Because these 
challenges are not adequately addressed by most early studies, it is difficult to draw strong 
conclusions from them. 3 

More recent studies, which have addressed some of these empirical challenges, suggest 
that annual per-student resource use tends to be higher in small schools than in large schools. 
For example, a study of small New York City high schools that were established before those 
we evaluate found that costs per graduate for schools with fewer than 600 students were about 
the same as those for large schools. However, because graduation rates were higher at the small 
schools, small schools have fewer enrollees than large schools with the same number of 
graduates. Thus, costs per enrollee were higher in small schools than in large schools (Stiefel et 
ah, 2000). Similarly, a study of exogenous enrollment changes at Indiana elementary schools 
(Kuziemko, 2006) found that decreased school size was associated with increased math scores 
and student attendance rates but also increased per-pupil costs. Another study of New York City 
high schools using data prior to most of the recent wave of high school refonns (Stiefel et ah, 
2009) is noteworthy because it controls for school fixed effects, which can help to reduce 
potential bias due to omitted or poorly measured variables and school inefficiency. This study 
finds that holding student needs and outcomes constant, annual costs per student increase 
monotonically as school size decreases. However, for themed schools, the analysis finds U- 
shaped cost curves that imply that costs per enrollee are minimized at a school size of roughly 
800 students. 

In addition to making descriptive comparisons of resource use in small and large high 
schools, the present paper takes a new approach to estimating small-school costs. It exploits the 
random assignment of incoming ninth-graders to SSCs or other high schools operated by the 
NYCDOE. This design switches the research question from “How do costs vary across different 


Efficient schools achieve a given level of outcomes for the least cost possible given their size, the wages 
they face, and the needs of their students. If a smaller (or larger) share of small schools than large schools are 
efficient in this sense, or if small schools on average are further away from (closer to) the efficient amount of 
resource use, then that will make small schools appear to have higher (lower) per-pupil costs than large 
schools, even if small and large schools can be operated in ways that achieve the same outcomes for the same 
per-pupil costs. Small schools might be operated more or less efficiently than large schools in a specific context 
for any number of reasons. For instance, the managerial abilities of principals in small schools may differ 
systematically from those in large schools. 

3 For a discussion of these issues see Andrews, Duncombe, and Yinger (2002). 
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schools, holding their outcomes and efficiency constant?” to “How much did a specific inter- 
vention cost, relative to a control condition, in order to achieve estimated improvements in 
student outcomes, given actual levels of efficiency?” This approach avoids the need to statisti- 
cally control for student outcomes or school efficiency, though it no longer provides a purely 
large- versus small-school comparison. 

This alternative approach to examining the costs of SSCs is particularly appropriate be- 
cause SSCs are not only smaller than the typical NYCDOE high school, but they also have 
other distinctive features that could influence both their costs and effects. For example, SSCs 
were created through a competitive proposal process that was designed to stimulate innovative 
education practices and ensure that SSCs promoted academic rigor and sustained personal 
relationships among students and faculty (see Bloom, Levy Thompson, and Untennan, 2010, 
for details). Also, in accordance with a key tenet of their initial design, SSCs were required to 
involve community partners as an integral part of their educational programs and often were 
created and operated with help from intermediary organizations. These intermediaries and 
partners might well provide staffing, support, and other resources to SSCs that are not generally 
available to other NYCDOE high schools. 


3. Sample and Data 

Of the more than 200 new, small high schools opened in New York City between 2002 and 
2008, 123 are academically nonselective and, in effect, determine admissions by a lottery if 
oversubscribed. Based on data for multiple student enrollee cohorts and follow-up outcomes, 
Bloom and Untennan (2014) report positive SSC effects for up to three cohorts of students that 
enrolled in oversubscribed SSCs between 2005-2006 and 2007-2008. For the present analysis 
we use five years of school expenditures data for each of the 8,521 students in the first two of 
these study cohorts (first-time ninth-graders in the falls of 2005 and 2006). This sample, which 
represents 84 of the original 123 SSCs, is the same as that used by Bloom and Untennan (2014) 
to estimate SSC effects on five-year high school graduation rates. 

The first set of analyses presented below, which we refer to as our “descriptive” anal- 
yses, compare per-pupil expenditures for this sample of 84 SSCs with per-pupil expenditures 
for a set of schools that we refer to as “other New York City high schools.” Since all SSCs 
begin with ninth grade, their comparison schools include all other NYCDOE schools that begin 
with ninth grade, except for schools with highly specialized missions (such as serving students 
who are failing in other high school environments) that differ from those of SSCs, and other 
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SSCs that were not included in the present sample. 4 This comparison group includes between 
225 and 295 schools depending on the school year being examined. 

The second set of analyses compares per-pupil expenditures devoted to students who 
randomly won the opportunity to attend an SSC on their high school choice list and subsequent- 
ly enrolled in one with the per-pupil expenditures devoted to students who randomly lost the 
opportunity to attend an SSC and as a result attended another New York City public high 
school. 5 These students who applied but were denied admission attended a total of 216 to 229 
schools depending on the year examined. 

Data for our cost analyses were drawn mainly from NYCDOE School-Based Expendi- 
ture Reports (SBERs) for the 2005-2006 through 2010-2011 school years. These data include 
per-pupil expenditure information by several functions for general education and special 
education students and are drawn from the NYCDOE’s accounting system of record. This 
system assigns all public school expenditures to specific schools, including funds that are spent 
directly by schools from their own budgets and funds that are spent for schools by district field 
offices and central offices. There is substantial variation across schools in the amount spent 
directly through school budgets. This variation reflects school differences in their number of 
teachers, administrators, and other staff members; school differences in the salaries of these 
staff members; and school differences in program offerings and compensatory aid. District-level 
expenditures are allocated to schools by a central cost accounting system. Some of these 
allocations are based on detailed resource usage information and others are made on a simple 
per-pupil basis. 6 

We also used data on individual teacher experience levels and salaries obtained from 
the Personnel Master File (PMF) maintained by the New York State Education Department, 
data from the NYCDOE School Construction Authority on facility capacity usage, and data on 
student enrollments and student-staff ratios from New York State School Report Cards. 


4 The following types of schools were excluded: schools that offer evening academic programs for high 
school students who are behind in credits, considering dropping out, or occupied by adult responsibilities in the 
daytime; schools devoted primarily to providing GED preparation; schools that exclusively serve special 
education students; residential programs; and schools that closed or initiated phase-out between 2005-2006 and 
2010 - 2011 . 

5 As discussed later, these experimental comparisons were the basis of an instrumental variables analysis 
that accounted for the fact that not all lottery winners attended an SSC and some lottery losers attended an SSC. 

6 The analysis below demonstrates that there is considerable variation in per-pupil expenditures, that this 
variation fits expectations, and that differences across schools in expenditures per pupil are similar to those we 
see in more direct resource measures, such as student-teacher ratios, all of which confirms that the SBERs 
reflect real differences in resource use across schools. 
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4. Cost Measures Examined 

The cost of SSCs can be divided into two types: start-up costs and ongoing costs. Start-up costs 
include resources devoted to planning and opening SSCs. Start-up activities were conducted by 
a wide range of actors both within and outside the NYCDOE. They include development and 
dissemination of guidelines for small-school proposals, provision of technical assistance to 
groups who were developing and submitting proposals, assessment and selection of proposals, 
site and facility planning, selection and hiring of SSC staff, procurement of equipment and 
supplies, and numerous other activities. The second type of cost involves the annual recurring 
costs of operating SSCs, which include costs of teachers, instructional support staff, pupil 
support services, administrative and instructional leadership, ancillary and building services, 
and the depreciation or lease of capital facilities. A complete accounting of these costs would 
include the value of ongoing resources provided by intermediary and partner organizations as 
well as those provided by the NYCDOE. 

Because they were drawn from so many different sources and were incurred some time 
ago, we could not assemble complete data on all SSC start-up costs. Hence the analyses we 
present focus mainly on the ongoing operating costs of SSCs. We do, however, return to the 
issue of SSC start-up costs after presenting our primary analysis. 

Following Levin and McEwan (2000), we define costs as the value of resources used in 
their best alternative use. The most comprehensive measure provided by the SBER of the value 
of resources used by specific schools is “direct service expenditures.” This measure includes all 
expenditures for services that were provided directly to students mainly in the school building 
during the school day and the school year. These expenditures include those recorded in each 
school’s spending plan plus district-level expenditures for services provided to each school or 
its students, including food, transportation, and building services. Any spending that is not 
included in direct service expenditures is allocated by the NYCDOE to schools on a per-pupil 
basis and thus cannot contribute to per-pupil cost differences among schools. 

Assuming that resources are purchased in competitive markets, expenditures for re- 
sources are usually good estimates of the value of those resources in their next best use (Levin 
and McEwan, 2000). The assumption of competitive markets, however, is potentially problem- 
atic in the cases of school staff and teachers, which represent the majority of school expendi- 
tures. Salaries for most school staff members are determined by collective bargaining agree- 
ments and vary substantially for individuals in the same position depending on their experience 
and education level. Thus, even between schools with the same number of teachers and other 
staff members, salary expenditures can differ appreciably. If salaries accurately reflect the value 
that school personnel have in their best alternative use, then salary expenditures plus benefits 
are an appropriate measure of the cost of these personnel. However, there may be reason to 
question this presumption. For example, a large literature on teacher effectiveness suggests to 
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many researchers that experience and education levels, which account for virtually all of the 
variation in teacher salaries, are only weakly related to teacher quality (Goldhaber, 2008; 
Rivkin, Hanushek, and Kain, 2005; Rockoff, 2004). Because SSCs tend to employ less experi- 
enced teachers who receive lower salaries than teachers in other high schools, per-pupil spend- 
ing in SSCs may understate the true value of teachers in SSCs and thus their true costs relative 
to teachers in other high schools. 7 

In order to remove this source of variation from our cross-school cost comparisons, we 
created a measure of “adjusted direct service expenditures per pupil” that values the time of all 
teachers at all high schools at the “district-wide” average high school teacher salary for the 
relevant school year. For details on how these adjustments were made, see Appendix B. 

We also developed a measure of direct student service expenditures that adjusts for 
school differences in student needs and teacher salaries. Table A. 1 in Appendix A compares the 
average shares of students eligible for special education services, English language learner 
services, and free lunches for SSCs and other NYCDOE high schools. These findings indicate 
that SSCs serve a somewhat smaller share of students with limited English proficiency. Howev- 
er, beginning in the 2007-2008 school year, they served a higher share of special education 
students. Throughout our analysis period, SSCs served significantly higher shares of free-lunch- 
eligible students. 8 

As a result of class size and other service requirements associated with special educa- 
tion, more resources are required to serve special education students than are required to serve 
general education students. For instance, the 2009-2010 SBERs indicate that schools spent more 
than three times as much per pupil for full-time special education students than for general 
education students. One consequence of this is that a significant amount of variation in expendi- 
tures across schools can be explained by the percentage of their students who are eligible for 
special education services. More generally, service needs vary for different types of students. In 
addition to special education students, English language learners and students from low-income 
families tend to require more services than do other students. 

If SSCs tend to serve more or fewer high-need, high-cost students than the typical high 
school in either of our groups of comparison schools, then simple comparisons of average per- 
pupil spending will either overstate or understate the relative cost of SSCs. In order to address 


7 Table 3 documents differences in teacher experience and salaries between SSCs and other New York 
City high schools. It is more likely that SSCs use less experienced, lower-paid teachers because they are new 
than because they are small. 

different types of special education students have different cost implications. It is possible that although 
SSCs serve a higher share of special education students, they serve a less costly mix of special education 
students. Data that we have been able to collect were insufficient to confirm or deny this possibility. 



this issue we used regression analyses to estimate the effects of student need differences on 
expenditures and used the resulting estimates to adjust our measure of per-pupil spending (for 
details see Appendix B). The difference between the average of this adjusted spending measure 
for SSCs and other New York City high schools indicates the difference in per-pupil spending, 
holding student needs and teacher salaries constant. 

Although they are the most comprehensive measures of resource use available, the pre- 
ceding three measures omit two sets of school resources that would be included in a complete 
accounting of ongoing school operating costs: facility space and resources provided by external 
partners. 9 We return to this issue later. 

In addition to the preceding resource use measures, we compared per-pupil expendi- 
tures for SSCs and other NYCDOE high schools with respect to five functions: classroom 
instruction; instructional support services; leadership, supervision, and support; ancillary 
support services; and building services. We also compared direct resource measures including 
student-teacher ratios and student-staff ratios, which were computed from PMF data or obtained 
from New York State School Report Cards. These comparisons were made to help shed light on 
why SSC costs differ from those for other NYCDOE schools. 


5. Descriptive Comparisons of School Resource Use 

Table 1 compares average direct service expenditures per pupil for SSCs and other New York 
City high schools. Because these averages are weighted by the student enrollment of each 
school, they represent the direct service expenditures for the average SSC student and for the 
average student in other NYCDOE high schools, respectively, for each of the years examined. 
Observed differences between SSCs and other high schools range from $678 per pupil in 2007- 
2008 to $1,411 per pupil in 2010-2011 (from 5.3 percent to 10.2 percent of spending in other 
NYCDOE high schools). These differences are statistically significant at conventional levels 
beginning in 2008-2009 and increase in magnitude over time. 

One might ask whether SSCs spend more per pupil because they are smaller than other 
NYCDOE high schools or for other reasons. Table 2 infonns this question by comparing direct 
service expenditures per pupil in high schools with various enrollments. Because findings from 
these comparisons are similar across all years, we present them only for the most recent school 
year, 2010-2011. Focusing first on other New York City high schools, we can see that schools 


9 SBERs do report lease payments for schools that lease space and expenditures on debt service. But many 
schools occupy space owned by the NYCDOE and the value of that space is not reflected in SBERs. Also, debt 
service is not allocated to schools in a way that accurately reflects differences in the value of the facilities they 
use. Expenditures for leases and debt service are not reflected in direct service expenditures. 
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with more than 2,000 pupils have considerably lower per-pupil direct service expenditures than 
do smaller schools, and schools with 250 or fewer students have considerably higher per-pupil 
expenditures than do larger schools. SSCs have lower per-pupil direct services expenditures in 
comparison with other New York City high schools of a similar size, although these differences 
are not statistically significant. These findings suggest that a primary reason that SSCs spend 
more per pupil than do other NYCDOE high schools is that SSCs are smaller, and smaller 
schools have higher per-pupil costs. 

Appendix Tables A.2 through A.6 compare per-pupil expenditures for SSCs and other 
NYCDOE high schools by functional categories that could be affected differently by economies 
of scale. For example, the literature suggests that high schools may have scale economies in the 
delivery of classroom instruction because small high schools may have trouble filling special- 
ized classes and thus have smaller than average class sizes for those subjects. Even larger 
economies of scale are expected for administrative and support functions because small schools 
often require administrative and leadership staffing that is comparable in size to that of larger 
schools. Results in the tables largely confirm these expectations. They indicate that SSCs spend 
slightly more per pupil on classroom instruction (between 0. 1 percent and 7.0 percent more) and 
substantially more per pupil on leadership, supervision, and support (between 18.6 percent and 
31.0 percent more) than do other New York City high schools. 

In addition to comparing direct expenditures per pupil, we used data from the Personnel 
Master File (PMF) and New York State School Report Cards to develop more direct measures 
of personnel resources used. Comparison of these personnel resource measures suggests that 
much of the additional cost of SSCs — and small schools more generally — is due to higher 
staff-student ratios. For instance, in 2010-2011, SSCs used 15.9 percent more teaching staff and 
32.9 percent more administrative staff per student than other NYCDOE high schools. 10 These 
differences between SSCs and other NYCDOE high schools are statistically significant and 
similar in other years. 

Teacher experience and salary data from the PMF are presented in Table 3. The com- 
parisons in Table 3 indicate that SSCs have maintained higher teacher-student ratios without 
spending significantly more on instruction (see Table A.2) because, on average, they hired less 
experienced (and thus lower-paid) teachers. The findings also indicate that differences in 
experience and salary are gradually eroding as SSCs age, which helps to explain why the cost 
disadvantage of SSCs as measured in Table 1 grows over time. 


l0 This difference implies that SSCs have about two fewer students per teacher than other NYCDOE high 
schools. 
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The top panel of Table 4 presents comparisons ofper-pupil direct service expenditures 
that are adjusted for differences in teacher salaries (as described earlier). As expected, because 
experience levels and salaries are relatively low in SSCs, adjusting for salary differences increas- 
es their observed cost differences from those reported for unadjusted measures in Table 1 . 

As noted earlier, SSCs serve higher percentages of students who are eligible for special 
education services and free lunches than do other NYCDOE high schools, although they serve 
lower percentages of English language learners. Thus findings in Table 1 and the top panel of 
Table 4 might either overstate or understate the relative costs of SSCs. To address this issue, the 
bottom panel of Table 4 presents cost comparisons that adjust for differences in teacher salaries 
and student needs (as described earlier). These findings indicate that direct service expenditures 
per pupil in SSCs exceeded those in other NYCDOE high schools by 7.4 percent in 2005-2006 
and 1.8 percent to 6.3 percent in subsequent years. 


6. Estimated Costs of Achieving the SSC Graduation Gains 

The preceding comparisons suggest that SSCs are more costly to operate than other, larger 
NYCDOE high schools, although the overall difference narrows when the level of student need 
or disadvantage is taken into account. However, these comparisons do not indicate the quantity 
of resources that were used to achieve the estimated effect of SSCs on their students’ graduation 
rates. For this purpose one needs to compare school costs for winners and losers of SSC 
lotteries, just as similar comparisons of high school graduation rates were used to estimate SSC 
graduation effects (Bloom and Untennan, 2014). 

The present section reports estimates of the value of school resources used for “target 
SSC enrollees” (defined below) relative to the value of school resources used for their “control 
group counterparts” (also defined below). The first step in this process was to assign our 
estimated school-year-specific per-pupil expenditures to each student in the sample that was 
used to estimate SSC effects on five-year high school graduation rates. This assigmnent of per- 
pupil expenditures was carried out for each of the five years after sample students entered ninth 
grade and was based on the high school in which they were enrolled during October of each 
school year (New York State’s enrollment month of record). For sample members who were 
identified as dropouts or graduates based on NYCDOE discharge codes, a school cost of zero 
was assigned for each year after the last year in which they were officially enrolled in a 
NYCDOE high school. 

This school cost assignment process resulted in very little missing data. Specifically, for 
the first, second, third, fourth, or fifth follow-up years, data were missing for only 4.8, 4.8, 4.5, 
6.3, and 15.7 percent of sample members, respectively. For the present analysis, these missing 
values were imputed using a single replication of a multiple-imputation model based on all 
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existing baseline and follow-up information for sample members. Alternative findings based on 
a sample that omits students with missing school cost values are very similar to those we report 
here and can be made available upon request. 

We then estimated the effect of SSC enrollment on high school costs per pupil using the 
statistical model employed by Bloom and Unterman (2014) to estimate the effect of SSC 
enrollment on high school graduation rates. This model uses lottery-based random assignment 
to an SSC or some other NYCDOE high school as an instrumental variable that predicts 
whether a student ever enrolled in an SSC during his five-year follow-up period. Predicted SSC 
enrollment is then used to estimate the causal effect of enrollment in an SSC on per-pupil 
expenditures. The approach produces an estimate of the average effect on school costs of 
enrollment in an SSC for students who enrolled because they won an SSC lottery. 11 This type of 
causal effect is typically referred to as a local average treatment effect on individuals who 
comply with their lottery assignment (that is, who attend an SSC if they win their SSC lottery 
and attend some other type of NYCDOE high school if they do not win their SSC lottery). 12 

Our statistical model was implemented using two-stage least squares to estimate the fol- 
lowing equations separately for each of five follow-up years and for the entire five-year period: 

First stage: SSC enrollment as a function of first SSC lottery assignment 

= Sy=i *j ■ Iji + Sy=i Yj ■ T i ■ hi + w i (!) 

where: 

E l = one if student i enrolled in an SSC at any time during his five-year follow-up peri- 
od and zero otherwise, 

Iji = one if student i participated in SSC lottery j and zero otherwise, 13 

T t = one if student i won his SSC lottery and zero otherwise, and 

Wi = a random error that is distributed independently and identically across students. 


11 See Bloom and Unterman (2014), pages 294-299 and 301-305, for a detailed explanation of the analytic 
procedures. As they describe, it is based on whether a student won his first SSC lottery, because given the way 
that high school assignment works in New York City, it is possible for a student to be in more than one SSC 
lottery (that is, for a less preferred SSC as well). 

12 See Angrist, Imbens, and Rubin (1996) for a discussion ofcompliers and local average treatment effects. 

13 Students participate in separate lotteries defined by the school, the year, and the SSC priority group the 
student belongs to, where SSC priority groups are defined by students’ residence and whether or not they had 
made themselves “known” to the SSC. As stated above, it is possible for them to be in more than one SSC 
lottery. For reasons discussed by Bloom and Unterman (2014), we (and they) only include students in their first 
SSC lottery (typically that for their most preferred SSC). 
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Second stage: High-school cost as a function of predicted SSC enrollment 

Y i = j: J J=1 aj-Ij i + 6-E i + e l (2) 

where: 

Y l = per-pupil expenditures (adjusted or not) at the school attended by student i, 

£) = the predicted value of SSC enrollment for student i from the estimated first-stage 

equation, and 

e t = a random error that is clustered by the school a student attends. 

Since assigmnent to treatment (7) ) is random only among students participating in the 
same SSC lottery, including lottery indicators (fy) is crucial for obtaining unbiased SSC effect 
estimates. It is also useful to specify a separate instrumental variable (instrument) for each 
lottery. These instruments were created by interacting the treatment indicator (7)) with the 
lottery indicators (7 ; j). The instruments are valid because they are randomly assigned and strong 
because they are highly predictive of SSC enrollment. 14 The estimated value of S is a consistent 
estimate of how many more (or fewer) school resources were devoted to target SSC enrollees 
than to their control group counterparts. 15 

Table 5 presents estimates of effects of enrollment in an SSC on direct service expendi- 
tures per pupil, with and without adjustments for teacher salaries and student needs. Following 
Bloom and Untennan (2014), we refer to students who won their SSC lottery and attended an 
SSC because they won this lottery as “target SSC enrollees.” We refer to students who lost their 
SSC lottery and attended some other NYCDOE high school because they lost their lottery as 
“control group counterparts.” Column one in the table presents estimated mean expenditures for 
target SSC enrollees, column two presents these estimates for control group counterparts, and 
column three presents the difference between these two estimates, which is the estimated effect 
of enrollment in an SSC. 16 

During each of the first four years of high school, direct service expenditures per pupil 
were neither substantially nor statistically different for target SSC enrollees and their control 
group counterparts. This finding holds for all three direct service expenditure measures. During 
the fifth year of high school, direct service expenditures per entering ninth-grader dropped 
precipitously for target SSC enrollees and their control group counterparts. This is because only 


14 Bloom and Untennan (2014) provide details on the first-stage results for Equation 1. 

15 Estimates of second-stage standard errors were adjusted to account for uncertainty in predicted SSC en- 
rollment, £j . 

16 Appendix C describes how estimates of per-pupil expenditures for target SSC enrollees were obtained. 
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a fraction of students in each entering-ninth-grade cohort enroll for a fifth year. However, fifth- 
year expenditures per entering ninth-grader were about one-third higher for control group 
counterparts than for target SSC enrollees ($5,063 versus $3,749). This is because fifth-year 
enrollment rates were about one- third higher for control group counterparts (26.4 percent versus 
19.7 percent). 17 This, in turn, reflects the fact that roughly one-third more control group coun- 
terparts were “available” for a fifth year of high school because they did not graduate in their 
fourth year. 

In sum, the quantity of resources devoted to SSC target enrollees is similar to the quan- 
tity devoted to their control group counterparts during the first four years of high school. 
However, because SSCs’ target enrollees are more likely to graduate in four years, they are less 
likely to require additional resources during a fifth year and thus use up fewer resources, on 
average, than the control group counterparts over the entire course of high school. This finding 
about average total student costs is in contrast to results presented earlier that indicate that 
average annual student costs are higher at SSCs than at other NYCDOE high schools. These 
results imply that the average control group counterpart attended a high school with annual per- 
student costs that were also higher than the overall average for non-SSC high schools. Possible 
reasons for this finding are that control group counterparts were substantially more likely than 
non-SSC high school students in general to attend high schools that had fewer than 250 students 
(16.0 percent compared with 1.3 percent) and substantially less likely to attend high schools that 
had more than 2,000 students (37.8 percent versus 51.9 percent). 18 

Bloom and Untennan (2014) report that five-year graduation rates for target SSC enrol- 
lees and their control group counterparts are 74 percent and 65 percent, respectively. Estimates 
in Table 5 of the present paper indicate that total five-year direct service expenditures for target 
SSC enrollees and their control group counterparts are $57,246 and $59,818, respectively. 
Together these results imply that the cost per graduate for target SSC enrollees ($77,780) is 16 
percent lower than the cost per graduate for control group counterparts ($92,454). If we use cost 
measures that were adjusted for teacher salary differences alone or for teacher salary differences 
and student needs together, corresponding results imply that the cost per graduate is approxi- 
mately 14 percent lower for target SSC enrollees than for their control group counterparts. 


n These rates were estimated using the impact model specified by Equations 1 and 2, replacing measures 
of expenditures as the dependent variable in Equation 2 with an indicator of whether or not a student was 
enrolled in his fifth year. 

18 The rates for control group counterparts were estimated using the impact model specified by Equations 1 
and 2, replacing measures of expenditures as the dependent variable in Equation 2 with an indicator of whether 
or not a student attended a school of a given size. 
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7. Additional Considerations 


The expenditure measures used in the preceding analyses do not reflect the value of three sets of 
resources used by SSCs: (1) facility space, (2) the value of human and financial resources 
contributed to SSCs by their external partners, and (3) the one-time, initial investment of 
resources devoted to creating each SSC. Although we do not have the data for a complete 
accounting of these additional costs, the following sections provide rough approximations of 
how their consideration might influence the conclusions drawn from the preceding analysis. 

Facility Utilization 

Specialized instructional spaces such as gyms, music rooms, science labs, etc. repre- 
sent fixed costs that can be optimally utilized only when school enrollments are sufficiently 
high. An important part of the SSC intervention, however, was to replace large high schools 
with multiple small high schools that were often located in the same building. The hope was 
that by sharing specialized instructional spaces in those buildings, small high schools could 
optimally spread the fixed costs of those spaces (New Visions, 2005). In fact, 73 of the 84 
SSCs in our sample shared a building with at least one other high school, 6 shared space with 
elementary or middle schools, and only 5 were located in stand-alone buildings. 19 A substan- 
tial number of New York City high schools other than SSCs share a building with one or 
more other schools, but a much higher fraction — nearly half — stand alone. So the question 
is whether sharing school buildings allows SSCs to use facility space as efficiently as schools 
with larger enrollments. 

Comprehensive measures that capture the quality as well as the quantity of space used 
by schools are not available. However, the NYCDOE School Construction Authority prepares 
an annual enrollment, capacity, and utilization report that includes a measure of facility capacity 
used by each school in the district. This measure is based on the number, size, and types of 
rooms used. For each room, the number of students that can be accommodated is estimated 
based on its size and function plus an assumption about how many periods per day the room can 
be used. 20 The resulting measure indicates the number of students that the space used by the 
school is capable of serving. Dividing this capacity measure for each school by its total enroll- 
ment in a given year provides an indication of how much capacity the school uses per student. 
A value less than one indicates that a school is overcrowded and a value greater than one 


19 These counts were tabulated by the authors using information from the NYCDOE Enrollment, Capacity, 
and Utilization Report for 2009-2010. 

20 It is assumed that regular classrooms can be used for seven out of eight periods per day and that specialty 
instructional spaces can be used for five periods per day. Specialty instructional spaces include music rooms, 
art rooms, computer labs, science labs, weight rooms, dance studios, and gymnasiums. 


15 



indicates some degree of underutilization, meaning that the school facility serves fewer students 
than it is capable of serving. 21 

Comparisons of capacity-enrollment ratios for SSCs and comparison high schools 
based on the 2010-2011 Enrollment, Capacity, and Utilization Report indicate that schools 
with small enrollments tend to use more facility capacity per student than do schools with 
large enrollments. Because SSCs tend to be smaller than comparison group schools, and 
smaller schools tend to use more capacity per pupil, more facility capacity is devoted to the 
average SSC student than to the average student in other New York City high schools. 
Specifically, SSCs use about 26 percent more facility capacity per pupil than other New York 
City high schools. 

Table 6 presents estimates of the difference in per-pupil capacity usage between SSC 
target enrollees and their control group counterparts (that is, the estimated effect of SSC 
enrollment on capacity usage per student). These estimates were computed like the correspond- 
ing estimates of SSC effects on direct service expenditures per pupil in Table 5 (by specifying 
capacity per pupil as the dependent variable in Equation 2). During years one, two, three, and 
four the ratio of school capacity to enrollment is greater than one for both the SSC target 
enrollees and their control counterparts. These results suggest that both sets of students tend to 
enroll in schools whose facility capacity is somewhat underutilized relative to NYCDOE 
standards. The facility capacity usage measures are well below one for both groups of students 
in year five because the majority of students do not enroll for a fifth year and students who are 
not enrolled use up zero facility capacity. 

Overall, the results in Table 6 suggest that SSC target enrollees use more facility ca- 
pacity per pupil than do their control group counterparts, but this difference is small. Over the 
full five years after students first enter ninth grade, SSC target enrollees use up only 6.3 percent 
more facility capacity per student than their control group counterparts, a difference that is not 
statistically significant. 

Converting these differences in facility capacity usage into dollar tenns is not possible. 
However, facility costs per pupil tend to be a small proportion of total school costs. For exam- 
ple, the 2010-2011 SBER indicates that only about 4.6 percent of expenditures by the 
NYCDOE were for debt service and leases for facility space. Because school buildings still 
have economic value even if they are not actively leveraged with debt, this figure is likely a 
substantial underestimate of the value of school facilities used in New York. Nevertheless, it is 
also likely that facility space is only a small fraction of annual school costs. Thus, if the small 

21 We were able to obtain data on these measures only for 2008-2009, 2009-2010, and 2010-2011. Howev- 
er, because these findings were similar across years, we assume that levels of capacity use in 2005-2006, 2006- 
2007, and 2007-2008 were the same as those in 2008-2009. 
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difference in facility capacity usage between SSCs and control group counterparts were ex- 
pressed in dollars, it is unlikely that the resulting amount would change the conclusions we 
reach through our analysis of direct service expenditures. 

Resources from External Partners 

The best available information about the value of contributions to SSCs from their ex- 
ternal partners is a survey of 13 SSC external partners conducted by New Visions for Public 
Schools. These survey results indicate that during the 2004-2005 school year, on average, 
external partners contributed financial and in-kind resources that were worth roughly $275,000 
per SSC (Soler-Mclntosh, Carrion, and Guntan, 2007). For a typical SSC with roughly 440 
students, this equals about $625 per student, which is only 5 percent of annual direct service 
expenditures. Thus adding these “off-the-book” costs for SSCs is unlikely to change conclu- 
sions about overall operating costs. Furthermore, since other NYCDOE schools can have 
external partners, only considering external contributions for SSCs would overstate the differ- 
ence in this cost component, which makes it even less likely that a full accounting of these 
costs, if possible, would change our conclusions. 

Start-Up Costs 

Human and financial resources from multiple sources were devoted to planning and es- 
tablishing SSCs. For example, the Bill & Melinda Gates Foundation (BMGF) planned to donate 
roughly $100,000 a year to SSC planning teams for their school planning year and the first four 
years of their school’s operation. Internal BMGF documents suggest that roughly $135 million 
went to support small schools initiatives in New York City between the fall of 2000 and the fall 
of 2007. How much of this funding was used either directly or indirectly to support the SSCs 
examined by the present analysis is hard to say. In addition, other foundations invested in this 
movement, and the NYCDOE allocated special funds and devoted district personnel to support 
planning efforts for small schools. Although a full accounting of these costs is not possible, it is 
important to note that they represent a one-time cost for each SSC that should be amortized over 
many years of its subsequent operation. Thus annualized costs per student are likely to be small. 
For instance, if we assume that (1) the $135 million from BMGF was split evenly across the 
216 small high schools created by the NYCDOE between 2002 and 2008, (2) schools have a 
useful life of 30 years, (3) the annual discount rate is 5 percent, and (4) average SSC enrollment 
is 440 students, then the annualized cost per SSC would be $40,657, or less than $100 per SSC 
student per year. Furthermore, if we assume that start-up resources provided by all other 
sources were equal to those of the Gates contribution, the additional costs would be less than 
$200 per student per year. Thus, it seems unlikely that consideration of these costs would 
substantially alter our conclusions. 
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Taken singly, consideration of facility usage, external resources, and start-up costs are 
unlikely to influence policy conclusions. Taken together, however, they might have more 
substantive implications. For example, they would enlarge somewhat the estimated annual per- 
student “cost disadvantage” of SSCs reported in Tables 1 and 4 relative to other NYCDOE high 
schools. In addition, they would reduce somewhat the estimated total per-student “cost ad- 
vantage” of SSCs reported in Table 5 relative to schools attended by control group members. 
Thus it is probably most accurate to say that there are no substantial differences in the total 
resources devoted to average target SSC enrollees and their control group counterparts. Howev- 
er, because of the substantially higher graduation rate for SSC enrollees, the average total 
cost per high school graduate is still likely to be substantially lower for SSC enrollees than 
for their control group counterparts. 


8. Implications for Policy 

Our findings indicate that annual per-pupil operating costs for SSCs are higher than those for 
other New York City high schools — particularly for high schools with over 2,000 students. For 
example, the difference between per-pupil direct service spending by SSCs and the typical New 
York City high school with more than 2,000 students in 2010-201 1 was $2,920, or 23.6 percent 
of average per-pupil spending in the large high schools. 22 This difference appears to reflect 
nonoptimal use of indivisible inputs such as school administrators and specialized teachers. 

However, control group counterparts in our analysis (1) also enrolled in schools with above- 
average resource usage, (2) were more likely than target SSC enrollees to attend a fifth year of 
high school, and (3) were less likely than target SSC enrollees to graduate from high school. 
Thus the present findings provide strong evidence that the cost per graduate for SSC enrollees is 
substantially less than that for their control group counterparts. These results are robust to 
different ways of estimating teacher costs, accounting for differences in student composition, 
and considerations of facility usage, start-up costs, and external provided resources. 

It is important to note, however, that neither the comparison of SSCs with other 
NYCDOE high schools nor the comparisons of resources devoted to target SSC enrollees and 
their control group counterparts provide a basis for assessing the costs of New York City’s 
small high schools initiative relative to the counterfactual situation of never having under- 
taken the small schools initiative. In the absence of this initiative, SSC students would not 
necessarily have attended the schools attended by their control group counterparts; many of 
them would likely have attended the large failing high schools that were closed as part of the 

22 This figure is based on a comparison of average 2010-2011 per-pupil direct service expenditures in SSCs 
from Table 1 and average 2010-2011 per-pupil direct service expenditures in other New York City high 
schools with enrollments over 2,000 from Table 2. 
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small schools initiative. Because it is not possible to identify what schools students would 
have attended in the absence of the small schools initiative nor to estimate the concurrent per- 
pupil costs of the large failing high schools that were closed, it is not possible to estimate the 
costs of the small high schools initiative relative to the counterfactual situation of never 
having undertaken the initiative. 

Nonetheless, combined with the results from Bloom and Untemian (2014), the present 
analysis provides internally valid, robust estimates of both the costs and benefits of a large 
sample of new, small schools relative to a well-defined control group. As a result, the analyses 
have important policy implications. To clarify these implications, it is worth distinguishing 
among three decisions that education policymakers might want to make. 

First, policymakers might need to decide whether an intervention in its current form and 
place ought to be continued. In this case, should the NYCDOE continue SSCs in their current 
form or seek to replace them with more cost-effective types of schools or programs? Of course, 
the answer to this question depends on what would be put in place of SSCs. Our findings 
suggest that the cost per graduating target SSC enrollee is 14 percent to 16 percent lower than 
that for their control group counterparts. Thus, replacing SSCs with schools like those attended 
by the control group counterparts is not warranted. 

The data do suggest, however, that placing SSC students in high schools with enroll- 
ments greater than 2,000 might reduce NYCDOE expenditures by approximately $12,000 per 
student over a student’s high school career. But these relative cost savings need to be weighed 
against the relative benefits of SSCs. For example, Rouse (2007) estimates that the average net 
present value of lifetime earnings gains produced by graduating from high school are between 
$300,000 and $635,000 in 2010 dollars. 23 If we assume that attending an SSC increases the 
probability of graduating from high school by 9.5 percentage points (as found by Bloom and 
Untennan, 2014), then SSCs generate income benefits that are between $28,500 and $60,000 
per entering ninth grader. This approximate lifetime earning advantage of SSCs is several times 
the approximate $12,000 cost disadvantage of SSCs relative to very large high schools in New 
York City. Furthermore, the effects of attending an SSC on high school graduation rates relative 
to attending a low-expenditure, large high school might be even greater than those relative to 
attending schools such as those of the average control group counterpart in our analysis. In 
addition, the benefits of graduating from high school are not limited to lifetime earnings gains 
(Belfield and Levin, 2007). 


23 Rouse (2007) actually reports between $260,000 and $550,000 in 2004 U.S. dollars, which are converted 
here to 2010 dollars using the Bureau of Labor CPI Inflation Calculator. 


19 



Thus, whether SSCs are compared with the types of high schools attended by control 
group counterparts or with the least expensive large New York City high schools, it appears that 
SSC benefits exceed their costs. Consequently, unless a superior alternative to New York City’s 
SSCs can be found, the present evidence suggests that they should be maintained. 

Second, officials might be interested in whether an intervention should be expanded in 
its current location. In this context, should the NYCDOE create more SSCs so that their benefits 
can reach more students? And, if so, how many additional SSCs should be created? These 
questions are more difficult to address. For example, additional SSCs might draw students from 
schools with lower per-pupil costs and thereby increase district costs. In order to be cost 
effective, the benefits of these new SSCs in terms of increased graduation rates for their students 
must exceed their additional costs. However, the estimated positive effects of SSCs on gradua- 
tion rates used for the present analysis apply to students who list SSCs as a school choice and 
who participate in an SSC lottery. Whether graduation effects for students who would be drawn 
into an expanded set of SSCs would be large enough to justify any increased costs involved in 
creating more SSCs is difficult to say. Thus, the marginal net benefits of adding more SSCs, 
while potentially large, are less clear than the average net benefits of existing SSCs. 

Third, policymakers might be interested in whether a similar school reform should be 
undertaken in other settings. In this case, should schools similar to SSCs be established in other 
urban districts? Few large-scale education refomis have been as rigorously evaluated as New 
York City’s SSCs and shown to be effective. Because SSCs generate positive net benefits 
relative to other New York City high schools, they represent a promising model. Whether SSCs 
would generate similar net benefits elsewhere (or benefits that are large enough to justify their 
costs) is likely to depend on a variety of factors. For example, the present analysis suggests that 
in districts with large high schools, the creation of new small high schools could increase annual 
school operating costs. Whether the benefits of new small schools will justify those additional 
costs depends on the effectiveness of current high schools and whether features of SSCs that 
made them effective, in addition to their small size, can be replicated. 

A number of urban districts have undertaken efforts to create new small high schools 
(Iatarola et ah, 2008; Wasley et ah, 2000), and other districts may be considering this model as 
part of their high school reform plans. However, evaluations of small schools refonns have been 
limited in their scope and methods and, to our knowledge, none of them has been subjected to a 
rigorous cost-effectiveness or cost-benefit analysis. This paper, in conjunction with previous 
research on SSC benefits (Bloom and Untennan, 2014), provides such an analysis for a large 
sample of newly created small high schools and finds that their costs per high school graduate 
are considerably less than those for schools attended by control group counterparts. In addition 
to encouraging innovation with small high schools elsewhere, we hope that these analyses help 
to provide a framework for how to conduct future evaluations of small schools initiatives. 
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Exhibits 




Small Schools of Choice 


Table 1 

Direct Service Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P- 

Value 

2005-2006 

71 

$11,499 

1,403.9 

225 

$10,560 

2,020.7 

$940 

505.54 

0.064 

2006-2007 

79 

$12,294 

1,315.6 

234 

$11,563 

2,022.6 

$731 

416.06 

0.080 

2007-2008 

84 

$13,399 

1,727.0 

240 

$12,720 

2,332.0 

$678 

428.88 

0.115 

2008-2009 

84 

$14,513 

1,526.5 

261 

$13,702 

2,335.0 

$811 

394.63 

0.041 

2009-2010 

84 

$14,835 

1,983.6 

276 

$13,679 

2,451.1 

$1,156 

404.17 

0.004 

2010-2011 

84 

$15,286 

1,875.9 

295 

$13,876 

2,760.5 

$1,411 

438.41 

0.001 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 
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Small Schools of Choice 


Table 2 

Direct Service Expenditures per Pupil, by School Enrollment, 2010-2011 


SSCs Other NYC High Schools Difference in Means 


Enrollment 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P-Value 

0-250 

4 

$16,038 

718.5 

33 

$18,962 

4,608.7 

-$2,925 

1,984.67 

0.150 

251-500 

71 

$15,462 

1,921.3 

110 

$15,599 

2,423.6 

-$137 

341.49 

0.688 

501-1,000 

9 

$14,115 

1,274.3 

77 

$15,258 

2,819.9 

-$1,143 

1,046.34 

0.278 

1,001-2,000 

0 

- 

- 

37 

$14,156 

2,122.9 

- 

- 

- 

Above 2,000 

0 

- 

- 

38 

$12,366 

1,935.3 

- 

- 

- 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 



Small Schools of Choice 


Table 3 

Teacher Experience and Salaries 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P- Value 

2005-2006 










Average years experience 

70 

5.92 

2.56 

218 

12.20 

3.34 

-6.28 

0.850 

0.000 

Percentage novice teachers 

70 

27.0 

17.6 

218 

8.5 

7.7 

18.6 

2.200 

0.000 

Average salary 

70 

$47,523 

3,691 

218 

$56,972 

4,804 

-$9,449 

1,222 

0.000 

2006-2007 










Average years experience 

78 

6.43 

2.52 

225 

12.38 

3.39 

-5.95 

0.712 

0.000 

Percentage novice teachers 

78 

20.9 

14.6 

225 

6.9 

6.5 

14.0 

1.599 

0.000 

Average salary 

78 

$53,519 

4,311 

225 

$63,638 

5,495 

-$10,119 

1,157 

0.000 

2007-2008 










Average years experience 

84 

6.75 

2.67 

232 

12.46 

3.34 

-5.71 

0.618 

0.000 

Percentage novice teachers 

84 

17.9 

11.5 

232 

6.6 

5.9 

11.3 

1.249 

0.000 

Average salary 

84 

$56,281 

4,835 

232 

$66,422 

5,702 

-$10,141 

1,062 

0.000 

2008-2009 










Average years experience 

84 

7.12 

2.63 

252 

12.70 

3.51 

-5.59 

0.600 

0.000 

Percentage novice teachers 

84 

12.7 

9.1 

252 

5.6 

6.5 

7.2 

1.199 

0.000 

Average salary 

84 

$61,655 

5,169 

252 

$72,248 

6,372 

-$10,593 

1,095 

0.000 

2009-2010 










Average years experience 

84 

8.09 

2.66 

268 

13.16 

3.58 

-5.08 

0.589 

0.000 

Percentage novice teachers 

84 

4.6 

5.7 

268 

2.2 

5.0 

2.4 

0.859 

0.006 

Average salary 

84 

$63,781 

5,144 

268 

$73,666 

6,457 

-$9,885 

1,066 

0.000 

2010-2011 










Average years experience 

84 

9.04 

2.54 

285 

13.51 

3.77 

-4.46 

0.599 

0.000 

Percentage novice teachers 

84 

3.4 

4.6 

285 

2.4 

4.7 

1.0 

0.763 

0.179 

Average salary 

84 

$65,817 

4,912 

285 

$74,332 

6,789 

-$8,515 

1,085 

0.000 


SOURCE: New York State Personnel Master File. 
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Table 4 

Adjusted Direct Service Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Adjusted for Diff erences in Teacher Salaries 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P- Value 

2005-2006 

70 

$12,071 

1,393.0 

218 

$10,494 

2,090.5 

$1,578 

529.5 

0.003 

2006-2007 

78 

$12,931 

1,298.4 

225 

$11,490 

2,060.2 

$1,441 

430.1 

0.001 

2007-2008 

84 

$14,010 

1,726.8 

232 

$12,579 

2,271.1 

$1,430 

419.8 

0.001 

2008-2009 

84 

$15,171 

1,595.8 

252 

$13,583 

2,333.0 

$1,588 

396.6 

0.000 

2009-2010 

84 

$15,437 

2,033.3 

268 

$13,495 

2,343.1 

$1,943 

389.4 

0.000 

2010-2011 

84 

$15,794 

1,946.4 

Adjusted 

285 

for Differences 

$13,664 2,581.6 

in Teacher Salaries and Studt 

$2,129 

nt Needs 

409.0 

0.000 

Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P- Value 

2005-2006 

66 

$11,718 

1,350.4 

213 

$10,914 

1,730.9 

$804 

450.7 

0.076 

2006-2007 

78 

$12,526 

1,351.7 

218 

$11,963 

1,530.8 

$563 

325.8 

0.085 

2007-2008 

84 

$13,490 

1,678.2 

226 

$13,062 

1,740.1 

$428 

329.0 

0.194 

2008-2009 

84 

$14,555 

1,320.8 

247 

$14,293 

1,588.6 

$262 

274.4 

0.340 

2009-2010 

84 

$14,751 

1,879.5 

256 

$14,214 

1,656.6 

$537 

285.8 

0.061 

2010-2011 

84 

$15,168 

1,684.2 

276 

$14,271 

1,941.6 

$897 

315.6 

0.005 


SOURCES: Author’s computations (described in Appendix B) using data from New York City Department of Education School-Based Expenditure Reports, 
New York State Personnel Master File, and New York State Report Cards. 
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Table 5 

Effects of SSC Enrollment on Direct Service Expenditures per Entering Ninth-Grader 



Target 

SSC 

Enrollees 

Control 

Group 

Counterparts 

Estimated 

Effect 

Estimated 

Effect 

(%) 

P- Value for 
Estimated 
Effect 

Direct Service Expenditures 





Year 1 

$11,936 

$11,956 

-20 

-0.2 

0.944 

Year 2 

$12,707 

$12,980 

-273 

-2.1 

0.293 

Year 3 

$13,935 

$14,405 

-470 

-3.3 

0.056 

Year 4 

$14,918 

$15,414 

-495 

-3.2 

0.070 

Year 5 

$3,749 

$5,063 

-1,313 

-25.9 

0.000 

Five-Year Total 

$57,246 

$59,818 

-2,572 

-4.3 

0.023 


Direct Service Expenditures Adjusted for Teacher Salaries 



Year 1 

$12,516 

$12,180 

336 

2.8 

0.241 

Year 2 

$13,288 

$13,189 

98 

0.7 

0.699 

Year 3 

$14,520 

$14,602 

2 

-0.6 

0.740 

Year 4 

$15,437 

$15,577 

-140 

-0.9 

0.617 

Year 5 

$3,810 

$5,106 

-1,296 

-25.4 

0.000 

Five-Year Total 

$59,571 

$60,654 

-1,083 

-1.8 

0.338 


Direct Service Expenditures Adjusted for Teacher 

Salaries and Student Needs 


Year 1 

$12,267 

$11,992 

275 

2.3 

0.310 

Year 2 

$12,957 

$12,951 

6 

0.0 

0.982 

Year 3 

$14,159 

$14,227 

-68 

-0.5 

0.765 

Year 4 

$15,060 

$15,268 

-207 

-1.4 

0.433 

Year 5 

$3,804 

$5,109 

-1,304 

-25.5 

0.001 

Five-Year Total 

$58,247 

$59,546 

-1,299 

-2.2 

0.223 

Sample Size 

3,544 

4,977 





SOURCES: Findings are based on data from the NYCDOE High School Application Processing System for eighth- 
graders in 2004-2005 and 2005-2006 plus data from the NYCDOE School-Based Expenditure Reports for school years 
2005-2006 through 2010-2011. 
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Table 6 

Effects of SSC Enrollment on School Capacity-Enrollment Ratios 
Five-Year Follow-Up Sample, Cohorts 1-2 



Target SSC 
Enrollees 

Control Group 
Counterparts 

Estimated 

Effect 

Estimated 
Effect (%) 

P-Value for 
Estimated 
Effect 

Year 1 

1.18 

1.07 

0.11 

10.2 

0.033 

Year 2 

1.17 

1.07 

0.10 

9.5 

0.030 

Year 3 

1.16 

1.06 

0.11 

10.1 

0.010 

Year 4 

1.14 

1.08 

0.06 

5.4 

0.108 

Year 5 

0.26 

0.34 

-0.08 

-23.9 

0.002 

Years 1-5 

4.91 

4.61 

0.29 

6.4 

0.102 

Sample Size 

3,544 

4,977 





SOURCES: Findings are based on data from the New York City Department of Education (NYCDOE) High 
School Application Processing System for eighth-graders in 2004-2005 (cohort 1) and 2005-2006 (cohort 2) 
plus data from the NYCDOE School Construction Authority's 2010-2011 Enrollment, Capacity, and 
Utilization Report for school years 2008-2009 through 2010-2011. 

NOTE: See page 16 of the report for more detail. 
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Appendix Table A.l 

Percentage of Students in Various Need Categories 


Special Education Limited English Proficient Free-Lunch Eligible 


Year 

SSCs 

Other NYC HS 

SSCs 

Other NYC HS 

SSCs 

Other NYC HS 

2005-2006 

9.48 

10.17 

6.57 

9.90 

71.03 

49.28** 

2006-2007 

10.86 

10.26 

6.01 

8.89 

69.50 

49.42** 

2007-2008 

12.41 

10.43** 

7.94 

11.71 

70.85 

49.92** 

2008-2009 

14.14 

10.98** 

7.46 

11.04 

69.56 

49.28** 

2009-2010 

15.59 

11.15** 

8.68 

12.07 

77.95 

62.10** 

2010-2011 

17.08 

12.14** 

9.51 

12.29 

78.31 

64.73** 


SOURCE: New York State Report Cards. 

NOTE: A two-tailed t-test was applied to the estimated effect. Statistical significance levels are indicated as: ** = 5 percent. 
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Appendix Table A.2 

Classroom Instruction Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P-Value 

2005-2006 

71 

$6,454 

979.7 

225 

$6,110 

1,051.3 

$345 

265.90 

0.196 

2006-2007 

79 

$6,843 

732.6 

234 

$6,638 

1,091.6 

$205 

224.79 

0.362 

2007-2008 

84 

$7,341 

1,119.6 

240 

$7,217 

1,287.7 

$124 

239.27 

0.606 

2008-2009 

84 

$7,685 

872.2 

261 

$7,627 

1,243.8 

$58 

210.97 

0.783 

2009-2010 

84 

$7,841 

909.0 

276 

$7,567 

1,307.4 

$274 

213.47 

0.200 

2010-2011 

84 

$8,148 

874.6 

295 

$7,617 

1,385.3 

$531 

219.20 

0.016 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 
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Appendix Table A.3 

Instructional Support Services Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P-Value 

2005-2006 

71 

$1,509 

413.4 

225 

$1,388 

431.1 

$122 

109.18 

0.266 

2006-2007 

79 

$1,612 

490.8 

234 

$1,561 

506.6 

$50 

106.38 

0.637 

2007-2008 

84 

$2,011 

516.9 

240 

$1,839 

577.0 

$172 

107.46 

0.110 

2008-2009 

84 

$2,232 

579.3 

261 

$1,998 

644.7 

$234 

111.28 

0.036 

2009-2010 

84 

$2,202 

644.3 

276 

$1,912 

711.9 

$289 

118.48 

0.015 

2010-2011 

84 

$2,312 

690.5 

295 

$1,999 

791.6 

$313 

127.90 

0.015 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 
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Appendix Table A.4 

Leadership, Supervision, and Support Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P-Value 

2005-2006 

71 

$1,718 

410.0 

225 

$1,311 

381.6 

$406 

97.27 

0.000 

2006-2007 

79 

$1,856 

510.4 

234 

$1,512 

423.8 

$344 

90.75 

0.000 

2007-2008 

84 

$1,987 

555.9 

240 

$1,675 

513.7 

$312 

97.39 

0.001 

2008-2009 

84 

$2,157 

519.4 

261 

$1,791 

534.6 

$365 

92.95 

0.000 

2009-2010 

84 

$2,203 

554.3 

276 

$1,828 

548.4 

$376 

92.30 

0.000 

2010-2011 

84 

$2,255 

487.1 

295 

$1,803 

631.8 

$452 

101.11 

0.000 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 
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Appendix Table A.5 

Ancillary Support Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P-Value 

2005-2006 

71 

$697 

118.9 

225 

$691 

154.5 

$5 

38.79 

0.889 

2006-2007 

79 

$771 

133.3 

234 

$762 

175.2 

$9 

36.26 

0.811 

2007-2008 

84 

$816 

142.3 

240 

$811 

194.0 

$6 

35.66 

0.877 

2008-2009 

84 

$868 

151.1 

261 

$854 

199.7 

$15 

34.03 

0.669 

2009-2010 

84 

$903 

151.2 

276 

$897 

217.8 

$5 

35.55 

0.879 

2010-2011 

84 

$892 

137.6 

295 

$914 

189.4 

-$22 

30.19 

0.462 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 
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Appendix Table A.6 

Building Services Expenditures per Pupil 


SSCs Other NYC High Schools Difference in Means 


Year 

N 

Mean 

SD 

N 

Mean 

SD 

Difference 

S.E. 

P-Value 

2005-2006 

71 

$1,082 

549.6 

225 

$1,026 

935.2 

$56 

233.07 

0.810 

2006-2007 

79 

$1,163 

566.8 

234 

$1,049 

705.5 

$114 

146.44 

0.437 

2007-2008 

84 

$1,156 

448.4 

240 

$1,096 

717.7 

$60 

130.93 

0.645 

2008-2009 

84 

$1,428 

852.8 

261 

$1,298 

735.3 

$130 

130.55 

0.319 

2009-2010 

84 

$1,579 

1,530.9 

276 

$1,374 

735.1 

$205 

144.45 

0.156 

2010-2011 

84 

$1,510 

1,486.7 

295 

$1,392 

888.5 

$118 

159.22 

0.459 


SOURCE: New York City Department of Education School-Based Expenditure Reports. 
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Appendix B 

Computation of Adjusted Spending Measures 




Direct service expenditures per pupil adjusted for teacher salary differences were computed as follows: 


1. Using data from the New York State Persomiel Master File (PMF), we computed for each year 
the average annual teacher salaries for each NYCDOE high school and for all NYCDOE high 
school teachers combined. We refer to the latter as the “district-wide” annual average teacher 
salary. 

2. We then used information from the PMF to determine the total number of full-time equivalent 
teachers at each high school. 1 Using these teacher counts, we created two estimates of total 
teacher salaries for each school. The first measure equaled the product of the number of full-time 
equivalent (FTE) teachers at the school times the district-wide mean teacher salary. The second 
measure equaled the product of the number of FTE teachers at the school times the mean teacher 
salary for that school. 

3. We then computed the difference between the first and second teacher salary measures for each 
school and added it to the school’s total direct student service expenditures. For schools with 
teacher salaries that were below the district-wide mean (like most SSCs), this adjustment 
increased estimated costs. For schools with teacher salaries that were above the district-wide 
mean, this adjustment reduced estimated costs. 

4. Last, we divided the adjusted total direct student service expenditures for each school by the 
school’s student enrollment. This created our first adjusted measure of direct service expenditures 
per pupil for each school. 

Direct service expenditures per pupil adjusted for teacher salary and student need differences 
were computed as follows: 

1. Using data for our sample of high schools, we regressed each school’s measure of per-pupil 
spending adjusted for teacher salary differences on the percentage of its students who were 
eligible for special education services, the percentage of its students who had limited English 
proficiency, and the percentage of its students who were eligible for free lunches. 

2. For each school, we then plugged its actual percentages of special education students, students 
with limited English proficiency, and free-lunch-eligible students into the estimated regression 
equation to compute its predicted spending per pupil. 

3. We then subtracted the predicted spending per pupil for each school from its actual spending per 
pupil. The resulting difference indicates how much more or how much less a school spends than 
what the predicted spending would be for a school with the same student composition (in terms of 
special education status, limited English proficiency status, and eligibility for free lunch). 


'Any teacher who is less than full-time is counted as 0.5 FTE. However, because only a very small percentage of 
teachers are less than full-time, the adjustments described here are not sensitive to how part-time teachers are treated. 
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4. We then added the difference computed in step 3 to the average per-pupil spending level for all 
schools in the sample. This last step rescales our adjusted measure of per-pupil expenditures to a 
level that is more similar to our unadjusted measure of per-pupil spending. 
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Appendix C 

Estimating Mean Expenditures for Target SSC Enrollees 
and Their Control Group Counterparts 




Table 5 reports estimates of mean per-pupil expenditures for target SSC enrollees and their control group 
counterparts. Because these two types of students cannot be identified individually, their mean 
expenditure levels cannot be observed directly. Instead, they must be inferred. This is made possible by 
the statistical properties of randomization plus two plausible assumptions. SSC lottery winners can be 
split into two main subgroups: those who enroll in an SSC (a large majority) and those who do not 
enroll in an SSC (a small minority who become no-shows). The model also portrays two main 
subgroups of control group counterparts: those who do not enroll in an SSC (a large majority) and those 
who do enroll in an SSC (a small minority who become crossovers). 

Assume (as seems plausible) that students are more likely to enroll in an SSC if they win an SSC 
lottery than if they do not win. Given this assumption, which is often referred to as “monotonicity,” or the 
absence of “defiers,” 1 there are two more subgroups in the model: “no-show counterparts” among control 
group members (control group members who do not enroll in an SSC and would not have done so if they 
had won their SSC lottery) and crossover counterparts among SSC lottery winners (lottery winners who 
enroll in an SSC and would also have done so if they had been randomized to the control group). 2 Lottery 
winners who are neither no-shows nor crossover counterparts are target SSC lottery enrollees. This is the 
subgroup of students for whom SSC effects were estimated, and it has a counterpart subgroup among 
control group members. These control group students who complied with their first SSC lottery plus 
target SSC enrollees are often called compilers in the literature. 

Randomization ensures that in expectation, (1) the proportion of SSC lottery winners who are no- 
shows (P NS ) equals the proportion of control group members who are no-show counterparts; and (2) the 
proportion of control group members who are crossovers (P co ) equals the proportion of SSC lottery 
winners who are crossover counterparts. Hence, the proportion of target SSC enrollees among SSC 
lottery winners equals the proportion of control group members who are their counterparts. 

Now assume that students winning an SSC lottery per se has no appreciable direct effect on 
future expenditures and that only by causing students to enroll in an SSC can their winning a lottery affect 
expenditures per pupil (which is often referred to as an “exclusion” restriction). 3 Consequently, 
randomization ensures that in expectation, mean expenditures per pupil for crossovers in the control 
group (Y co ) equal those for crossover counterparts among SSC lottery winners. 

Now note that the mean value of per-pupil expenditures for all SSC lottery winners who enroll in 
an SSC (Y lwe ) is a weighted average of mean per-pupil expenditures for target SSC enrollees (Y tarE ) and 
crossover counterparts with weights equal to the relative size of each group. Then note that the observed 
mean expenditures per pupil for crossovers ( Y co ) is an unbiased estimate of mean expenditures per pupil 
for crossover counterparts. Together, these facts imply that: 

1 See Angrist, Imbens, and Rubin (1996). 

2 

“Angrist, Imbens, and Rubin (1996) refer to no-shows and their control-group counterparts as “never takers’’ and to 
crossovers and their treatment-group counterparts as “always takers.” 

3 

See Angrist, Imbens, and Rubin (1996). 
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Ylwe 


1_ p NS~ p CO y 

i -Pns tarE 


+ 


p co 

-1 -Pns- 


l co 


(C.1) 


Solving Equation C. 1 for Y tarE yields 


^ tarE 


1-Pjvs 
Li-Pns-Pco- 


l LWE 


p CO 

L ^-Pns-Pco- 


' CO 


(C.2) 


In this way, the implied value of Y tarE can be inferred from observed values of P NS , P co , Y LWE , 
and Yco- Findings in column two of Table 5, which are estimates of the mean unadjusted and adjusted 
direct service expenditures per pupil for control group counterparts, are obtained by subtracting the 
estimated effect of enrolling in an SSC in column three from the estimated mean unadjusted and adjusted 
direct service expenditures per pupil for target SSC enrollees in column one. 
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